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ABSTRACT 

This comparative study was conducted in preparation 
for selecting a successor to the National Teachers Examination (NTE) 
for initial certification of Montana educators. The paper begins by 
discussing the background of using teacher competency tests in 
Montana, and examining demographic data on Montana students and 
certified teachers. Then, the Basic Skills and Pedagogy Tests of the 
Content Mastery Examination for Educator (CMEE), offered by National 
Computer Systems, and two versions of Praxis T Academic Skills 
Assessments and Pre- Professional Skills Tests (PPST) offered by the 
Educat ional Testing Service, are compared . The two sets of tests are 
examined in terms of five questions: (1) the knowl edge , skills, and 
concepts the tests measure; (2) cost to new teachers; (3) whether 
these tests should be offered on computers and how would that impact 
costs; (4) w tat must be done to validate and implement uhe new test; 
and (5) why Montana is mandating a teacher competency test. A concern 
expressed is the need to include American Indians in the devel opment 
and validation of the tests. The findings of the comparison indicate 
that there is more reliance on tests that demonstrate utilization of 
knowledge, such as essays to assess writing, and less reliance on 
tests with discrete bits of knowledge. Only one company offers a 
multiple choice pedagogy test, but with the caveat that there is no 
assurance that the person who passes the test will be a good teacher. 
Based on the concept that a teacher's performance will have to be 
evaluated when he/she is actually teaching, the Praxis Series has 
added a performance test to assess teaching ability. Finally, since 
the ETS has removed all deadlines associated with phasing out any NTE 
tests, the results of the study suggest that Montana koep the NTE 
Core Battery in place while continuing to evaluate the Praxis Series. 
(Contains 13 references.) (ND) 
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A Comparison of the 

Content Mastery Examination for Educators (CMEE) 

and the 

Praxis I Academic Skills Assessments 
for 

Initial Certification of Montana Educators 
by 

Claudette Morton, Ed.D. 



Background: 

Educational reform efforts in the 1980s brought into existence a new form of 
testing-teacher competency tests. While Montana often resists jumping on national 
bandwagons, with regard to teacher testing, this was the exception. In the mid-eighties 
the Board of Public Education held hearings on the need to hpve such a test. The 
arguments for a test were mainly that if all the other states had a test and Montana didn't, 
teachers who couldn’t pass tests in other states would come to Montana to get 
certification. Some felt having a test would improve the image of teachers, since other 
professions, i.e., law, medicine and accounting, require a test for licensure. All felt it 
would provide a threshold of knowledge which all beginning teachers should have. The 
professional groups supported the idea of a test as long as their current members were 
grand-fathered in. John Kohl, then Dean of Education at Montana State University- 
Bozeman, made a compelling argument in support of the National Teachers Examination, 
even though the Office of Public Instruction (OPI) had recommended another test to the 
Board. The Board ultimately adopted, at the request of Dean Kohl, the National 
Teachers Examination (NTE) Core Battery from the Educational Testing Service (ETS). 
In the rules that the State Board adopted, they grand-fathered in current practicing 
educators except for those who changed class of certificates. In other words, if teachers 
became administrators, moving from a Class II to a Class III certificate, then they had to 
take the test, but if they stayed in the classroom for the next twenty years they did not. 
However, if an educator allowed his or her certificate to lapse, then part of reinstatement 
included successful passage of the NTE. 

How did the state of Montana know what successful passage was, or what would 
be the qualifying scores? The state conducted a validation study directed by Dr. Alan 
Zetler, fonner Dean of Education at Western Montana College. Dr. Zetler's design for 
validation followed an accepted format (3) and is sighted in national studies of teacher 
test validation (Popham 287). With input and direction from an advisory forum, he 
convened a statistically representative sample of the then current teaching force and 
asked them to examine the test battery to detennine (a) content review: the degree to 
which there was a valid match of items with those components of teacher preparation 
programs in Montana, (b) job relevance: the pertinence of the items to the job of a 
beginning teacher, and (c) knowledge estimation: the establishment of recommended cut 
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scores for each of the three tests in the battery (Zetler 13). Based on this validation 
study, the qualifying scores were set for Montana. They were as they are today: 648 m 
communication skills, 644 in general knowledge, ar., 648 in professional knowledge 
(Board of Public Education, 10*838). 

It is interesting to note as one looks at the administrative rule which put the NTE 
in place that no rationale or explanation for this rule is given. Administrative rule 
10.57.211 simply begins: 

(1) Effective July 1, 1986, all new applicants for initial class 1, 2 or 3 
certification must provide evidence of having completed the national teacher 
examination core battery with a minimum score established by the board 
(10-837). 

It then goes on to require those who allow their certificates to lapse to have to meet tnis 
new standard, to grandfather in the existing educators and to give the qualifying scores. 
This is in strong contrast to the other administrative rules regarding student assessment. 
10'" 101 Student Assessment (10.811) and 10.55.603 Curriculum Development and 
Assessment (10*773), in which the first part of each rule provides a rationale for the State 
Board enacting the rule. Perhaps, by omission, what is unspoken, but clear and verified 
by talking with those who were involved in the implementation of the NTE, was that 
requiring passage of the NTE Core Battery was by no means viewed as a measurement 
of one’s ability to teach, nor does ETS, the test publisher, make that claim. It was instead 
viewed somewhat politically and, as has been previously stated, was put in place as a 
threshold of knowledge which all beginning teachers should have. 

Since then, at least five of the education programs at Montana's institutions of 
higher learning have incorporated the communication skills and the general knowledge 
tests from the Core Battery into their teacher education program admission standards. 
Currently, the NTE is given three times a year, at nine Montana sites. 

In 1989 the Certification Standards and Practices Advisory Council of the Board 
of Public Education began a study to examine the impact of the NTE on American Indian 
teacher education students. Ors. Gerald Bekker of Montana State University-Northern 
and Harry Lee of Montana State University-Billings worked with OPI’s then Director of 
Teacher Education and Certification, Dr. John Voorhis, and the author to determine why 
American Indian teacher education students were having more difficulty passing the NTE 
than their White counterparts. Special tests were administered to American Indian 
students at the two institutions. The results were analyzed through an item analysis to 
pinpoint the kinds of knowledge on which Native Americans did well and knowledge areas 
with which they had difficulty. Based on that work and later study, the Board of Public 
Education ultimately adopted amendments to the original administrative rule on teacher 
testing to allow an exception for educators who had difficulty passing certain parts of the 
test. The exception allowed the substitution of an adequate grade in an appropriate 
college course for a qualifying test score. This was a realization by the State Board that 
no single test score should ever be a closed gate to state licensure. 
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Generally, since the implementation of the test, the National Teachers Examination 
Core Battery has done its job. The problem is that the Educational Testing Service 
informed the Board that it was phasing out the NTE and moving to a series of new tests-- 
The Praxis Series. Because this is a new assessment, its implementation, along with 
alternatives, must be examined. However, in an examination of alternatives, it is 
important to note that the field of assessment has become much less isolated. No longer 
should the determination of a test be made without taking into consideration why it is 
given and to whom. Therefore, in order to examine the alternative tests and give an 
appropriate evaluation, it is important to look at the teaching force in Montana and the 
student body of Montana Public Schools. 



Montana Information: 

In the 1993*94 school year, according to the Office of Public Instruction statistics, 
there were 163.020 students enrolled in Montana's public schools. Of that nurriber 88 
percent were White, 10 percent were American Indian, 0.8 percent were Asian, 1.4 
percent were Hispanic, and 0.5 percent were Black. [See Table 1.] With regard to how 
well Montana students <lo, the traditional measurement has been how well do they pass 
tests, and again according to the Office of Public Instruction Facts about Montana 
Education they do very well. Montana students were first in the nation on the Armed 
Services Vocational Aptitude Battery (ASVAB): this test measures both practical 
vocational and academic skills for those who wish to pursue military service and receive 
technical training by the armed services; and Montana students were first on the eighth 
grade National Assessment of Educational Progress Mathematics test (2). According to 
an article in the September/October issue of Montana Schools, Montana students’ scores 
on the 5iAT were nine percent above the national average, and the one percent of 
Montana American Indian students who took the SAT tost had average scores 1 7 points 
higher in verbal and 28 points higher in mathematics than the national average for 
American Indian students (6). Those Montana students taking the ACT, about three 
times as m&ny as took the SAT, were five percent higher than the national average (6). 
Clearly by traditional measures of success on standardized tests, Montana students are 
doing well. 

How does the teaching force compare to the student body? For the 1993-94 
school year, according to research conducted by Dr. Dori Nielson, analyst at OPI, there 
have been 3,125 initial teaching certificates granted in Montana in the last eight years 
(from 1986 the year the NTE was put in place, to 1994). Of that number 96.4 percent 
have been given to White teachers, 3.0 percent have been given to American Indian 
teachers, 0.4 percent to Aslan teachers, 0.1 percent to Hispanic teachers and 0.3 percent 
to Black teachers. As can be seen from Table 1, there are some discrepancies in the 
certified staff of Montana’s schools mirroring the student body. 
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*1993-94 Certifieo Staff • initial Montana Certification in past 8 years. 



Also, according to Dr. Nielson’s analysis, still using the teaching ^orce initially 
certified over the last eight years, there are 175 Montana educators with provisional 
certification who have not successfully passed the NTE. Of that totai, 87.4 percent are 
White and 1 1 oercent are American Indian. The other 1 .6 percent is not accounted fur. 

Further comparison of the teaching force to the student body of Montana, again 
using Or. Nielson’s work, reveals that 46 percent of all Montana students are female and 
52 percent are male. However, the teaching force is 67 percent female and 33 percent 
male. Of that total 66 percent of all White teachers are female and 34 percent are male, 
but 73 percent of American Indian trachars are female and only 27 percent male. [See 
Table 2.] Looking at the further analysis of Montana’s student body, according to the 
enrollment data for 1993-94, the American Indian student body exactly mirrors the total 
Montana student body with regard to gender (Keenan 5). The reader may be concerned 
that these figures for teachers only represent the last eight years of certification. 
However, when one looks at Dr. Nielson’s work analyzing those initially certified over the 
last eighteen years, which is more representative of the total teaching force, then the 
percentages are exactly the same for the total certified staff by gender and the White staff 
mirrors that total exactly, but the female American Indian staff increases to 73 percent 
and the male American Indian staff decreases to 27 percent. Clearly, if modeling is 
important in education, then this is an issue to be examined further. 
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TABLE 2 



Certified Staff in Montana by Gender 
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In any examination of the success rate of a test, it is important to know how many 
teachers came from in-state and how many from out-of-state. According to Dr. Nielsen’s 
work, of those who received initiai certification in the past eight years, 67 percent came 
from in-state and 33 percent came from out-of-state. Further breakdown shows that the 
White teachers almost mirrored this figure; 66 percent came from in-state, with 34 percent 
from out-of-state, but many more of the Amencan Indian teachers came from in-state 
[See Table 3.] 



TABLE 3 

Source of Degrees of Certified Staff 
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*1993-94 Certified Staff - initial Montana certification in past 8 years. 



Finai'y, it would seem appropriate to know how Montana teaching candidates have 
done on the NTE. Based on results sent to OPI from ETS for Montana first time test 
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takers between October 1992 and Summer 1993. it can be seen [Table 4] that the failure 
rate is generally low or in an acceptable range except in regard to American Indian tes* 
takers. It should be emphasized that these are first time test takers, and the percentages 
do not reflect the final passage rate. 



TABLE 4 

Failure Rates of First Time Test Takers. Oct 92 to Summer 93 
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Given this information it would appear that any analysis and comparison or study 
of teacher competency tests to replace the NTE must examine how the new tests have 
dealt with or utilized American Indians in their development and/or validation. 



General Comparisons of CMEE and Praxis I Assess ments: 

The purpose of this section is to compare the Basic Skills and Pedagogy Tests of 
the Content Mastery Examination for Educators (CMEE) offered by National Computer 
Systems (NCS) with the two versions of Praxis I Academic Skills Assessments; Computer 
Based Academic Skills Assessments and Pre-Professional Skills Tests (PPST) offered 
by the Educational Testing Service (ETS). 

In terms of general psychometric features, it is this author’s opinion that Dr. Ernest 
Rose, your other testing consultant, did an excellent job in his analysis for the Certification 
Standards and Practices Advisory Council and the Board of Public Education. In general 
this author concurs with Dr. Rose’s findings and would just like to reiterate some of those. 
Both tests are criterion referenced, though ETS prefers to use the term "objective- 
referenced.- In the general breakdown of standardized tests on a continuum from norm 
referenced to criterion referenced both CMEE and Praxis I fall into the latter category. 
In other words, both tests do not base scores on how groups of students have done as 
in the case of a norm referenced test, but determine acceptable scores on the concept 
of mastery at a certain level of a body of knowledge determined to be important to the 
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field. While both companies have good rationale for strong content validity, neither claims 
any predictive validity. That is, the successful passage of the tests is not an indicator of 
how successful the test taker will be as a teacher. Both tests have quite acceptable 
reliability according to material provided by the tests' manufacturers, and both were 
created by acceptable methods used in the development of large scale assessments. 

In order to truly analyze these tests as a replacement for the NTE Core Battery 
there are a variety of issues to be examined. In order to know to whom and for whom 
the test is being given, the portrait of the Montana student body and current teaching 
force has already been drawn. Other considerations to be explored include; 

I. What knowledge, skills and concepts do CMEE and Praxis I; 
Computer-Based or PPST measure? 

II. What will the cost of the new test be to prospective new teachers? 

III. Does Montana want to move its teacher testing into a new format 
and offer the test on computer? If so, how much will this add to 
the costs and can the current testing centers accommodate 
computer testing? 

IV. Do the tests take into consideration the cultural differences in 
Montana's teaching force and public schools’ student body? 

V. What must be done to validate and implement the new test 
in Montana? 

VI. Finally, why is the state mandating a teacher competency test? 



At this point it would seem appropriate to take the key questions one by one and 
respond to them. 



I. What Do the Tests Measure? 

Praxis I measures proficiency in reading, writing and mathematics. The Content 
Mastery Examinations for Educators also has a Basic Skills Test which measures reading, 
writing and mathematica. In addition, CMEE offers a multiple-choice test on pedagogy. 
Neither ’PST nor Praxis I includes this component. (It should be noted that the 
Educational Testing Service has developed a pedagogical test as part of the Praxis 
series, although it is not a multiple choice test, but rather a performance type of 
assessment to be given to student teachers or practicing teachers. There are also 
optional subject-specific pedagogy modules in the Praxis II assessment and ETS is in the 



process of developing a pedagogical test which might be suitable for Montana's purposes, 
but it will not be available until March 1995. (However, there will be no further discussion 
of these assessments because it was agreed that this analysis would be confined to pre- 
professional general instruments.) The CMEE Basic Skills test is one test composed of 
40 reading items. 20 writing items plus an essay and 40 mathematics items. The CMEE 
Basic Skills Test requires three to three and one/half hours for completion. The time limit 
is optional and can be set by each state. Praxis I. the computer version, is made up of 
three separate tests. The Reading test has 36 items and takes 90 minutes to complete, 
the writing test has 35 computer-delivered questions and one essay question and takes 
66 minutes to complete, and the mathematics test has 29 questions and takes 55 minutes 
to complete. Praxis I PPST includes three separate basic skills tests. The Reading Test 
has 40 items and is given in a 60-minute time period, the Writing Test has 45 multiple- 
choice questions and one essay question given in 60 minutes, and the Mathematics Test 
includes 40 items given in 60 minutes. [See Table 5.] 



TABLE 5 

Comparison of Number of Items and Time Requirements 
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Each of these tests, as their names imply, measures basic skills in reading, writing 
and mathematics that beginning teachers should have at their command. For a more 
complete examination of the skills, concepts and knowledge covered by each test see 
Table 7. It is important to examine and compare these content outlines. At this point it 
might be instmctlve to explain that there is a high correlation among the SAT, the ACT 
and all basic skills teacher competency tests. The reader may well be wondering why not 
just use the scores from the college entrance tests. The indication here is that both the 
CMEE and Praxis I have undergone significant work both in their design and their 
validations to determine basic skills that beoinnino teachers need . The underlined phrase 
is the critical language. There has been no standard setting process nor any content 
validity work done on the SAT or the ACT as teacher competency tests and there could 
oe legal problems using them for this purpose. 



II What Will Be the Cost of the New Tests? 



According to information provided by both test publishers, the tests vary somewhat 
See Table 6 for a complete comparison. As can be seen from the table CMEE s Basic 
Skills Test is the least expensive and Praxis I: Computer-Based Academic Assessments 
the most expensive. While this author would not encourage that a decision about which 
test to give be based on cost to the new teachers, the issue of cost to a profession where 
last year teachers in state were paid as little as $11,309 (Morton 1) should not be 

excluded. 



TABLE 6 

Cost Comparison of Tests 
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•All 3 parts 

III Does Montana Want to Move Its Teacher Test ing into a Different Format? 

Both CMEE and the Praxis I: PPST are offered in the multiple-choice with four foils 
(options) format. This is the traditional model of test taking. Both include an essay 
question to assess writing which must be answered in the student’s own handwriting and 
language. The Praxis I Computer-Based tests utilize a variety of test taking formats. 
These assessments include a diversity of Items: traditional multiple choice with four foils, 
modified multiple-choice Items, i.e., choosing two from a list of ten, ordering and even 
some open-ended Items. In addition, the test is given, as its name implies, on a 
dedicated computer. The change in the format of the items on this third test makes it 
appealing to those who wish to see assessment alternatives and who believe that the 
field of education should reflect the latest thinking in this field in its own assessments. 

There is, of course, another issue. It is often said that what we test is what we 
value, and there is an effort as part of the reform movement in American education to 
integrate technology. If all this is true, then it would seem appropriate to have prospective 
teachers take the test on the computer. The concerns with requiring the test be taken 
by computer are not small. Do the current test centers have the space and the 
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eauipment needed? According to ETS. Montana already has two centers, in Billings and 
Helena where the computer version of Praxis I can be taken. Upon further investigation 
It was found that these two centers are not at college campuses, but at Sylvan Learning 
Centers Given that the testing centers have generally been located on college 
campuses, where budgets have been tight for the last several years, the additional 
expense must be considered. It might be possible to dedicate some funds from teacher 
certification fees to be used to assist the campus testing centers in obtaining the 

According to Stanley J. Kalisch. Jr., the ETS Field Marketing 
Representative. ETS can fund the necessary items and have a pay back system. To 
offer the Computer-Based Praxis I. a test center needs to have a dedicated room, specific 
carrels, and a minimum of four or five dedicated 486 computers tied to a server. These 
are all aspects to be considered seriously. 

What if someone is not computer literate? Can alternatives be provided? Can the 
PPST be substituted for the Computer-Based test? Is this something to be phased in 
over a certain time period? These questions were asked as part of a structured interview 

with Mr. Kalisch of ETS. Mr. Kalisch responded that first, can 

a tutorial with it. so people need not have many computer skil s. Second, the PPST can 
be substituted for the Computer-Based version so the computerized test can be Phased 
in. Given the complexity of adopting the Computer-Based version, it is still important to 
point out that it is a more open format which is more state of the art in 
circles. Finally, if this is a test to be given to improve the professional 'mage of educators 
and if technology is a significant tool of American education, then should it be ignored in 

assessment? 

IV no the Tests Taka into Consideration the Cultur a l Differences in Montana’s Teachin g 
Fnrcft and Public Schools' Student Body? 

Much has been written on the need to have role models for children and the 
importance of their having teachers with whom they can relate. Eighty-eight percent of 
Montana’s children are White and they can easily relate to their 96.4 percent Whi e 
teachers It's true that we could wish for a better balance between male and female 
teachers so that our boys could see more examples of men in education as well as 
women but that is improving. However, what about the ten percent of the American 
Indian students who see only three percent of their teachers who look like themselves? 
That is an issue. In the background of this paper, I began with the descnption of the 
student body and the teaching force in this state. It was pointed out that there was a 
special study done in regard to American Indian students’ passage of the current teacher 
competency test which resulted in alternatives being allowed to substitute for successfu 
passage of the test. Still, it is important to see how the test makers in the creation of 
ihese various instruments which are the subject of this comparison utilized American 
Indians in the development and the validation of these tests. Also, it would ^ 
interest to note if any Montana educators or a rural category was used in the test 

development or validity studies. 



10 




13 



TABLE 7 
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With regard to the development of the content determination for CMEE, no 
Montanans were used in either the Basic Skills Test (Information Services Division 20) 
or the Pedagogy Test (21). As far as minority representation, American Indians were not 
identified as a specific group so this author assumed they were included as "Other" in that 
area. According to NCS, there was one "Other" on the content-determination panel for 
the Basic Skills test and no "Other" on the Pedagogy test (20-21). On the bias review, 
no American Indians were even mentioned in the discussion (29): however, in giving the 
results of the national field test American Indians were shown to have been 1 .8 percent 
(35). No data were reported specifically to them. There was no mention of any sort of 
a breakdown in review among rural, suburban and urban educators. However, educators 
were broken down by regions for the item appraisal procedures (31). 

In terms of the effort by ETS to include a rural component, Montanans and 
American Indians in the development and validation of their new Praxis Series, these 
elements were present. To determine what the content should be of the new tests, a 
survey was sent to 6000 random educators and specifically to 1 ,500 Black and Hispanic 
educators. Additional surveys were sent to professional organizations. ETS received 
almost 3000 useable surveys (Rosenfeld & Tannenbaum 7). On the survey to include 
a skill or concept the item had to receive a rating of 3.5 or higher on a 5 point scale by 
all groups (9). One of the groups included was a rural component (19). In a later 
multistate study of aspects of validity and fairness of items, six Montanans were on the 
panel (only Colorado had more and they had seven) (The ETS Validity Subcommittee IV- 
6). In that same study there were four American Indians on the mathematics test, two 
on the reading test and three on the writing test of the Praxis I series (IV-5). In fact, six 
percent of all panelists for this latter study were American Indians. 

Because of Montana's past history with the NTE and its American Indian students, 
this author specifically asked representatives of each company, in the structured 
interview, what they were prepared to do to accommodate American Indian test takers. 
In a phone interview with Darcy Thomas of NCS on October 24, she said that there had 
not been a statistically significant American Indian population in the development of the 
CMEE Basic Skills or Pedagogy Test. However, these tests are criterion-referenced and 
represent mastery of content knowledge and skills important for beginning teachers. She 
suggested that we could do a pilot test with teacher education students from that 
population. She also advised that with the CMEE tests, the state can determine the time 
allowed for each test. NCS recommends between three to three and one/half hours for 
each test, but the determination of a time limit and what it would be is up to the individual 
state. She said that they have developed a comprehensive study guide which expands 
the content guide, provides a study test, and goes through why the answers are right or 
wrong. She felt that both of these features, the test time flexibility and the new study 
guide, would be of great assistance to improving the success of the American Indian 
students. 
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In a similarly structured phone inten/iew with Stanley Kalisch of ETS in regard to 
this issue, he responded that ETS has determined that beginning January 1. individuals 
who have English-as-a-second-language will be able to apply for fifty percent extended 
time on all ETS tests. They will have to submit documentation at the time of application 
to take the test, but they will be accommodated in a special test. 



V. What Must Be Done to Validate and Implement the New Test in Montana? 

Again this was part of the structured interview with officials from NCS and ETS. 
According to Ms. Thomas of NCS she said that because there has already been a 
national validation of CMEE, there would be no need to do a specific validation in*state. 
It is critical, of course, that the state do its own standard setting, in other words determine 
its cut score in a legally defensible manner. NCS would assist the State Board to help 
them bring together a panel of twelve to fifteen appropriate experts, including practicing 
teachers, from Montana to do this. As far as the actual implementation, NCS would work 
with the state to determine the appropriate number of test centers looking at a balance 
between the number of sites and the number of people being tested at each site. If the 
state were to adopt CMEE for its teacher competency test, individual administration 
schedules are set up with each state. Ms. Thomas said that NCS would create a 
registration bulletin for the state of Montana with specific information on Montana's testing 
policy and how it might fit with teacher education requirements. She concluded that the 
whole process could be done and the new CMEE test could be "up and running in a short 
time." 



According to Stanley Kalisch of ETS when asxed about validation of the Praxis I. 
he said that because Montana has had the NTE, Montana would be considered a 
transition state. Since this is the case, ETS would pay for the standard setting study for 
the PPST. There would not need to be a separate study for the computer version of 
Praxis I because ETS has a concordance table to translate the PPST scores into 
appropriate ones for the Computer-Based assessment. The standard setting study would 
consist of convening for one day a group of professionals, the majority of whom would 
be practicing teachers. Mr. Kalisch said that it would be important for the panel to 
represent not only Montana's ethnic and gender diversity, but also diversity in teaching 
fields and levels and he believed that the number would be about thirty for a good panel. 
During that day long meeting which ETS would manage, the participants would judge the 
validity and set the qualifying scores including determining sample writing scores. If the 
state wished to set a compensatory model, that is a total score where students could do 
better on some of the tests and worse on others but still come out with an acceptable 
total, then the meeting would have to continue into the next day. Mr. Kalisch said that 
there were definite pros and cons on utilizing this latter version of qualifying scores. He 
concluded that ETS would write up the report from this state study and have it back to 
Montana within two months. 
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As far as implementation is concerned. PPST is given seven times a year and 
could be given in existing testing centers, though probably not all the centers would use 
all the testing dates, The computer version can be given anytime. 



VI. Why Is the State Mandating a Teacher Competency Test 

This question has been initially answered in the background section of this paper. 
Still, it is a question which this author would like to invite the Council and the Board to 
reconsider. When the NTE Core Battery was implemented in Montana, it was with three 
reasons in mind: 

a. To improve the image of the field of teaching as a 
profession; 

b. To protect Montana’s public schools from becoming a 
dumping ground of inadequately educated teschers 
who could not pass a teacher test elsewhere; 

c. To set a threshold of knowledge which all beginning 
teachers should have. 

It is very difficult to determine if the image of Montana's educators has improved 
in the last eight years; their pay hasn't. Montana teachers were ranked forty-second in 
pay in 1991-92 (Morton 10). Perhaps that is not a fair measure, but short of a survey of 
public attitudes, it is the only measure we have. However, this author would submit that 
having only a test of basic skills for teacher licensure would not only be unprofessional, 
it would be demeaning. 

What about the concerns that Montana may be a dumping ground if it doesn’t have 
a test for teacher licensure when so many ether states do test teachers? According to 
information provided in a letter from Catherine Havrilesky of ETS, "45 states now use 
some form of teacher testing* for prospective candidates. Given that real world 
information, there may need to be some tempering of the ideal expressed in the previous 
paragraph However, what is not known is how the states which don’t require teacher 
testing are screening their candidates for licensure and if they are facing the problem of 
becoming a dumping ground. 

Finally, is there a need for a "threshold of knowledge" assessment? Either of 
these two tests will serve that purpose. If in fact it is the judgment of the Council and the 
Board to adopt one of the two tests, then it is the hope of this author that the answers to 
the questions raised in this paper will provide some guidance for their final determination. 
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Conclusion: 



In the period of time that the author has been working in the field of assessment 
(sixteen years), there has been a real shift in the purpose of tests. In the world of 
assessment, there is much less reliance on tests with items of discrete bits of knowledge 
and more on demonstration of utilization of knowledge: hence, the addition of the essay 
in all of these test formats to assess writing. There is also generally a belief that 
assessment, even national standardized assessments, should exhibit some relationship 
to curriculum. In Montana the curriculum for teacher education is outlined in the Montana 
Teacher Education Program Standards. The instruments which have been evaluated for 
this report reflect only a very small segment of th^'se standards. The assumption is that 
students come into the college teacher education programs with those basic skills, and 
most Montana teacher education programs have admittance requirements which bear out 
this assumption. 

As to the pedagogy tost, only one company, NCS, now offers a multiple choice 
format test. Even with this tost, NCS is careful to point out that there is no predictive 
validity claimed; that is there is no assurance that the person who passes the test will be 
a good teacher, only that ho or she can answer questions correctly about teaching. ETS 
has made a bold gamble with the development of the Praxis Series and including in it an 
assessment based on the concept that to determine if a person is going to be a good 
teacher, that person’s performance will have to be evaluated when the person is actually 
teaching. The reason for this is the same rationale that was used when the test makers 
went to essays to evaluate writing. It has become very clear since 1974 when the first 
National Assessment of Educational Progress Writing Assessment was given, that if you 
want students to loam to write, and you want to evaluate how well they write, then you 
must ask them to write in an assessment. Now with the move in the assessment world 
to performance and portfolio assessment, we are beginning to be able to measure how 
well learners can accomplish other complicated tasks. Whether or not we are at the point 
of being able to develop a performance assessment which can measure the act of 
teaching, the field will have to decide, and it is not the purview of this paper. What is 
clear is that a paper and pencil knowledge test will not tell us whether a music teacher 
can excite fifty middle school students and open up the world of ensemble work to them: 
it will not tell us whether a science teacher can help her American Indian students see 
the relationship between their natural world and the world of chaos theory: nor will it tell 
us whether or not a first grade teacher can help all the children in his class not only learn 
to read, but love to read. 

There is one bit of new knowledge which the author has just received which should 
be considered. In a letter from Stanley J. Kalisch, Jr., Field Marketing Representative for 
the Praxis Program, dated October 18, 1994, in which he encloses a copy of a letter to 
Peter Donovan, of your staff, he states: 
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"In response to its clients who have been using the NTE and who are not 
ready to make changes to newer Praxis tests, ETS has removed all 
deadlines associated with the phasing out of any NTE tests. Only as client 
demand ceases for a test, does ETS expect to eliminate a test. " 

This is welcome news. It gives the Council and the Board time to see how the new 
Praxis I Computer-Based series works out in other states, and though no comparison was 
done in this report between CMEE and the NTE Core Battery, it would seem it would not 
be worthwhile to shift to another test which is so similar to the current one. since any 
change to a different test will create expense and disruptions. Given this latest 
information from ETS. my final recommendation is to keep the NTE Core Battery in place. 
Examine carefully the PRAXIS I Computer-Based Academic Skills Assessment and even 
consider down the road the PRAXIS Ill-Performance Assessment when it has had some 
real world experience. Consider Montana's own alternative to a teacher competency test 
developed through the wisdom of the Board of Public Education with guidance from the 
Certification Standards and Practices Advisory Council, the Office of Public Instruction 
and the teacher education programs at Montana's colleges and universities. Whatever 
change is ultimately made, if there is a professional test for teacher licensure, it should 
truly be a professional test. 

One Final Author's Note: These conclusions and recommendations may be 
frustrating because they are not simple and definitive, but any good research on any 
subject should lead to further questions as well as some answers. I hope I have provided 
you some answers, but not so comfortably that the Council and the Board will not 
consider further investigation in this area. 
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